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I If next generations of heavy quarks and leptons exist within the SAi , they 

can manifest themselves in Higgs boson production at the Tevatron and the 
LHC, before being actually observed. This generation leads to an increase of 
the Higgs boson production cross section via gluon fusion at hadron colliders 
. by a factor 6-^9. So, the study of this process at the Tevatron and LHC 

can finally fix the number of generations in the SAi . Using the WW* Higgs 
boson decay channel, the studies at the upgraded Tevatron will answer the 
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Abstract 



' question about the next generation for mass values 135 GeV < Mh < 190 



GeV. Studying the rr channel we show its large potential for the study of the 
Higgs boson at the LHC even in the standard case of three generations. At 
the Tevatron, studies in this channel could explore the mass range 110 — 140 
GeV. 
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I. INTRODUCTION 



The Standard Model {SAi ) does not fix the number of fermion generations. So far it is 
not known why there is more than one generation and what Law of Nature determines their 
number. 

The studies on the Z peak at LEP proved that there are exactly three generations of 
quarks and leptons with light neutrinos. However, the existence of next generations with 
heavy neutrinos is not completely excluded. The S'-parameter value obtained from modern 
electroweak data |l| makes the fourth generation disfavored but only at the level of 2.5 a 
accuracy. Anyway, new quarks and leptons should be heavier than the t-quark. 

At a first glance, one cannot determine the number of generations in the SAi before a 
direct observation of these new particles. Here we propose a simple way how to determine 
the number of such families before their direct discovery provided the simplest variant of the 
SAi (with one Higgs doublet) is valid. The key is given by experiments probing the Higgs 
boson H production via gluon fusion at hadron colliders (Tevatron and LHC). In accordance 
with modern data, we have in mind that Mh > 95 GeV 0. We assume that the masses of 
new quarks S> Mh- 

The proposal stems out from well known facts (see e.g. 0). In or pp collisions, the 
Higgs boson is produced mainly via gluon fusion. The production cross section is propor- 
tional to the two-gluon decay width of the Higgs boson T{H gg). This width is induced 
by diagrams with quark loops (see Fig. |1|). The dominant contribution to T{H —>■ gg) 
comes from heavy quark loops. Indeed, the amplitude corresponding to such a loop is 
oc (^ga^/maxlMji/, nig} (here denotes the strong coupling constant). The Yukawa cou- 
phng constant gg between the Higgs boson and the quarks is gg = rug/v (with v = 246 GeV — 
the Higgs boson v.e.v.). At rUg ^ M^, the quark contribution is finite and mg-independent. 
(The light quark contribution is suppressed as ~ {nig/ MhY .) 

Therefore, in our discussion we restrict ourselves to the t-quark and the possible fourth 
generation. Since the new fermion generation contains two extra quarks, the amplitude of 
this decay increases by a factor about 3 and the two-gluon width — by a factor about 9 for 
a light enough Higgs boson. 



In one-loop approximation the two-gluon decay width of the Higgs boson can be written 

as 



The quantity $ is the sum of loop integrals <l>g corresponding to different quarks q: 



II. THE TWO GLUON WIDTH AND GLUON FUSION 
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With reasonable accuracy we consider only the t-quark and Nq very heavy quarks from next 
generations (assuming that they are much heavier than H). In this case 

$ = -2rt[l + (1 - rt)x\rt)] - ^iV, . (2.3) 

(If the heavy quark masses are comparable with Mh, all numbers have to changed in a 
similar way as the effect of the t-quark mass seen in Table |l| at Mh > 200 GeV.) 

In the following we denote by T^g and the two gluon width and the production cross 
section in a model with k generations, for = 4 we have Nq = 2. The discussed dependence 
of the two-gluon width T^g on the Higgs boson mass M^ in one-loop approximation is 
shown in Table |I[ (The effects of radiative corrections are considered below.) Note that for 
a Higgs boson mass close to the ti threshold, the relative contribution of the top quark in 
Eq. ( p.3|) grows, which leads to a decrease of the Ygg/T^g ratio. 

The cross section for the Higgs boson production in pp (pp) collisions is given as the 
convolution of the cross section for the subprocess gg ^ H with the gluon structure functions 
gi^x^Q"^ = W"^) (W^ = X1X2S is the total two-gluon cm. energy squared). Since the 
total Higgs boson width is less than 10 GeV for Mh < 300 GeV and the distribution of 
colliding gluons is smooth, the standard narrow-width approximation CTgg^H = ^^r^g5(iy^ — 
Mh)/{8Mh) is used with high precision. With this form of agg->Hi the cross section of 
interest reads 

a{pp —> H + . . .) = 

TT^ r 1 dXi ( M^^nfT - /M-2N (2.4) 

The relative enhancement of the Higgs boson production due to the fourth generation is 
simply 

a\pp^H + ...) _Tlg 
a'{pp^H + ...) T%- 

The essential radiative corrections are of the following two sources: 

i) QCD corrections to the two-gluon width and the production mechanism. Those correc- 
tions have been calculated for the SAi with three generations in Ref. 0]. They enhance 
the Higgs boson production cross section ( p.4| ) at the LHC by a factor K = 1.5 1.7. 



Varying the Higgs boson mass from 100 GeV to 1 TeV the i^-f actor increases almost 
monotonously in that region. It changes weakly for heavier quarks. 

ii) Strong Yukawa interaction of the Higgs boson with heavy quarks. The scale of these 
corrections is given by g'^/lAn)^ = m^/(47rf)^. So, they can be neglected for heavy 
quark masses < 3 TeV. (For example, at < 1 TeV these corrections are less 
than a few percent 0.) For still larger quarks masses our results can be considered as 
a first rough estimate. 



Therefore, the factor K should be added in the cross section ( |2.4| ). In accordance with 
the results of Ref. 0] and the previous discussion, we choose the value = 1.5 for both the 



3 



Tevatron and the LHC and at all Higgs boson masses. A more precise value of K would 
change our cross sections (and in particular the ratio ( p.5|) ) by less than 10 %. 

Using the parameterizations of the gluon structure functions from Ref . , we obtain the 
Higgs boson production cross sections for different decay channels at the Tevatron (Fig. ^ 
and the LHC (Fig. ^ in the SM. with three or four generations. To test the sensitivity 
to the parameterization of the gluon structure functions we have calculated the same cross 
sections using an alternative parameterization from Ref. |]^. We observe that the obtained 
cross sections differ weakly from those presented in the Figures. 

In the Higgs boson mass range 100 — 140 GeV the rf decay channel for the three 
family model has a branching ratio varying from 0.08 {Mh = 100 GeV) to 0.04 {Mh = 
140 GeV). For the four family model the corresponding branching ratio reduces from 0.065 
to 0.036. At Mh > 150 GeV the effect of the fourth generation changes the observable 
branching ratios only marginally. The main background for this channel is given by the 
process qq Z{pf) rf. It is also shown in Figs. ^ and |^. 

The effect of the next generation can also be seen at the LHC using the process gg ZZ 
much above the Higgs resonance position, at nig ^ Mzz ^ Mh 0. The main contribution 
is given by the diagram with the s-channel Higgs intermediate state. The ggH vertex 
entering in the measurable cross section is rrig independent (just as for the real Higgs boson 
production) and the amplitude is Mh independent at Mzz ^ Mh- For Higgs boson masses 
Mh < 190 GeV the smaller cross section (far from the resonance) is compensated by the 
opportunity to use the ZZ final state with low background. The resulting non-resonant 
effect is observable at the LHC assuming a large enough luminosity integral. 

III. THE TWO PHOTON WIDTH AND THE RELATED CROSS SECTION 

The fourth generation also modifies the two-photon decay width of the Higgs boson. 
This decay originates from similar loops with leptons, quarks and W bosons. Here the W 
boson and t-quark contributions (other loops give negligible contributions) are of opposite 
sign and the former dominates strongly at Mh <^ 2m(. Thus the fourth generation results^ 
in a significant reduction of the two-photon width (see Table 

However, at Mh ^ 2mf (where the t quark contribution exceeds the W boson contri- 
bution) a new fermion generation enhances the Higgs boson two-photon width again. This 
trend can be also seen in Table |I[ The QCD radiative corrections are small in this case . 

From the behaviour of the two-photon width we conclude that the fourth generation 
would result in a strong (but destructive in a wide range of masses Mh) effect for the Higgs 
boson production at photon colliders (see Fig. 

Concerning hadron colliders, this decay channel is of specific interest if the Higgs boson 
will be discovered via the 77 decay at the LHC. Note that at Mh > 100 GeV the cross 
section of the process — > if — > 77 , being proportional to Br^ , increases if the fourth 



We included both heavy quarks and a charged lepton, assuming that their masses are much 
larger than Mh- 



4 



generation exists (Table (Here Br^ is, as usual, the branching ratio for the decay channel 
0.) The influence of an extra family becomes stronger with increasing Higgs boson mass. 



IV. SEARCH STRATEGIES AT THE TEVATRON AND THE LHC 

For a known luminosity integral C and detection efficiency of a considered decay channel 
e we calculate below the significance of an effect as the ratio S/ a/B with the number of 
events for the signal S and the background B. The number of events for the signal with 
k generations is given by Sk = eC cr^ignnh the corresponding number for the background by 
B = eC (Tbkgd- Both S and B are calculated with necessary cuts. In our calculations we use 
the integrated luminosity £ = 30 fb~^ for the upgraded Tevatron and £ = 100 fb"^ for the 
LHC. 



A. Different decay channels 

The hh decay channel. At Mh < 135 GeV the dominant decay channel is hh. 
The signal to background ratio S/B can be easily estimated since the main contribution 
to the non-resonant background is given by the hb production in the same gluon collisions. 
Therefore, it is sufficient to compare the non-resonant hh production with that occurring in 
the Higgs boson decay ignoring the particular distribution of the gluon flux. Using the just 
mentioned procedure to calculate the cross sections, we find that the ratio S/B < 0.001 even 
in the case of four generations (more than 10^ events are necessary to obtain a significance 
S/\fB > 3). As a result, the bb channel cannot be used for our problem. 

The TT decay channel. In spite of the relatively low branching ratio for this 
channel, the relative value of the background is not so huge. The possibility to use the rf 
decay channel for discovering the M.SSM. Higgs boson at the LHC and SSC was considered 
in Ref. ||^. A more detailed study for the LHC has been performed recently in Ref. |Tr!|. 



When searching for a single r lepton in pp collisions, its leptonic decay channel should 
be used which has the branching ratio 1/8.5. Just this value of the detection efficiency was 
achieved in the CDF experiment [|11|. For the rr pair the leptonic channel for one r and 



the hadronic jet channel for the another r can be used 1^,0]. This procedure results in a 
branching ratio rf — >■ Iv + jet of 0.09. However, realizing the presence of other background 
processes, various sources of detection imperfection etc., we prefer to use a twice lower value 
e = 0.04 to obtain cautious estimates. Using suitable cuts we hope that the resolution 
AM ^ 10 GeV is achievable in the effective mass M of the produced rf system. In the 
calculations we use a cut-off p± > 30 GeV in the transverse momenta of the produced r 
and a rapidity cut-off |?7| < 1.5 for the Tevatron and I77I < 2.5 for the LHC. 

We consider here the main source of a "physical" background — the process qq 
Z{j) rf. The cut-off in p± keeps 80 % of the signal and 72 % of the background at 
Mh = 100 GeV, 90 % of S and 85 % of B at Mh = 140 GeV. The rapidity cuts leads to 
88 — 94 % of the signal and 76 — 85 % of the background (depending on the Higgs boson 
mass) and improve the S/ ^J~B ratio by 1 — 2 % for the Tevatron. At the LHC these cuts 
keep 84 — 90 % of the signal and 66 — 71 % of the background (depending on Mh^ and 
improve the Sj \/B ratio by 4 — 7 %. From these estimates it follows that both cuts do not 



5 



cause any significant improvement of the S/yB ratio. However, these cuts are still useful 
because they cut off background processes of other origin. 

Figs. ^, ^ and Table |T| show the corresponding signal and background cross sections 
at the Tevatron and the LHC (with a common i^-factor K = 1.5). The last columns in 
that Table show the significance S/ a/B defined above (both three and four generations are 
considered in the case of the LHC) with a detection efficiency e = 0.04, a luminosity integral 
30 fb~^ for the Tevatron and 100 fb~^ for the LHC and the cuts mentioned above. One 
can see that the effect of four generations will be observable at the Tevatron in the Higgs 
boson mass range Mh = 110 — 140 GeV. At the LHC the effect will be huge for a standard 
one-year luminosity 100 fb^^. Going from the Tevatron to the LHC the improvement of 
the ratio S/ ^/B is very natural, since with decreasing of x ~ / y/s the gluon structure 
function increases much faster than the quark structure functions. 

More detailed calculations can either improve these estimates or to make them worser. To 
this end additional backgrounds like ti — > W~^W~bb — >• rf +. ..—>■... should be considered as 



it was made in Ref. Those backgrounds need detailed simulations with some additional 
cuts (for example, the anticoincidences with 6-quarks can be useful). The relative value 
of this background is essential at the LHC |T^. At the Tevatron it corresponding value is 
much smaller since the ratio Ivtitj \fs is larger, resulting in a much lower cross section for 
the ti pair production. Furthermore, in our calculations we have averaged the cross sections 
over an effective mass interval 10 GeV. Detailed studies should answer, whether this mass 
resolution is achievable or whether it it should be improved. 

The \x~ decay channel. At the LHC this channel can be used as an addi- 
tional opportunity to search for a new generation. The branching ratio of this channel is 
{mr/m^Y ~ 283 times smaller than that for the rf channel. However, the mass resolution 



here can be 1 GeV or even better |T2[, the signal is very clean, the detection efficiency is 
close to 1 and the background is reduced by a factor > 10. The last column in Table |Tl| for 
the LHC shows the Si{^)/\fB value for the channel with a mass resolution of 1 GeV. 

The "yy decay channel. This channel is also proposed for the Higgs boson study 
at the LHC ||14| . The accuracy needed to detect extra generations can be seen from Table ||. 

The VFVF* decay channel is dominant at Mh > 135 GeV. The simulation 
of the process gg ^ H WW* with background was performed within the SA4 with 
three generations in Ref. [|l^]. It was shown that the Higgs boson can be excluded at 95 % 



confidence level at 135 GeV< Mh < 190 GeV (which means that S/yB > 2) with a total 
luminosity integral £ = 30 fb~^. Since the signal in our case increases by a factor p = 8.2-=-8.5 
with an unchanged background, the luminosity integral needed for the same significance is 
reduced by to less than 0.5 fb~^. Enhancing £ by a factor r, the S/^/B ratio increases 
by ^/r. Therefore, in the case of four generations a luminosity integral of 3 fb~^ is sufficient 
to see the Higgs boson in this mass interval with S/\/B > 5. An additional simulation is 
necessary to find the lower bound of the accessible mass interval for the luminosity integral 
30 fb-i. 

Fig. 1^ shows that the opportunities at the LHC are significantly richer. 

The ZZ decay channel. At Mh > 190 GeV this channel is best suited for the 
Higgs boson search. The value of the signal with reasonable kinematical limitations can be 
estimated using the results of Ref. ^ as input. For the resonance production the signal is 
~ Mzz/{MhTh) times larger than that calculated in Ref. p. This factor is larger than 100 
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at Mzz > Mh and Mh < 250 GeV. In accordance with the numbers of Table |T|, the ratio 
of products of gluon fluxes varies quickly with Mh'- very roughly it decreases by a factor 
of the order of 100 from the LHC to the Tevatron in this mass region. Therefore, with a 
production cross section being 8 times larger (Table |ID the effect is expected to be sizeable 
even at the Tevatron (run III). The opportunity to observe the effect at lower values of Mh 
(the ZZ* channel) should be also explored. 



B. Different colliders 

Studies at the upgraded Tevatron. Summarizing the previous discussion, 
we conclude that the direct Higgs boson production in the rf channel for the Higgs boson 
mass interval 110 — 140 GeV and in the WW* channel for Mh = 135 — 190 GeV will 
answer the question about the next generation for the Higgs boson mass interval 110 — 
190 GeV. This statement is beyond doubts for the WW* channel, and it should be tested 
in more detail for the rf channel. These studies could reveal the existence of the fourth 
generation even if the Higgs boson is not discovered in the associative production. 

Studies at the LHC. At the LHC the Higgs boson is expected to be visible in 
different channels depending on its mass [IT3. In all cases, strong signals of the fourth 



generation provide the opportunity to study the problem of new heavy generations in the 
SA4 even if the new particles are so heavy that they cannot be directly produced. Using 
the fi~^fi~ channel with a possible mass resolution of about 1 GeV and with a very clean 
signature is also possible despite the very low branching ratio. 

Besides, one can hope that the rf channel can be used to observe the Higgs boson via 
gluon fusion even for the SAi with three generations in the Mh interval 100 — 150 GeV. 
Comparing the data in the rf , WW*, ZZ* (ZZ) and 77 decay channels will be essential to 
test the coupling constants of the Higgs boson with different particles and thus verify the 
Higgs mechanism of the mass generation in the SA4 . If Mh < 190 GeV, the non-resonant 
gg ^ H* — s> ZZ process H] will be a cross check of the results obtained at the Tevatron. 



V. CONCLUDING REMARKS 

Certainly, the presented numerical estimates are rough. Detailed Monte Carlo simula- 
tions should take into account specific features of the detectors (to account for the detection 
and triggering efficiency in more details). They will show the exact regions of the Higgs 
boson mass where the effect of the fourth generation could be seen in particular channels 
at the Tevatron and the LHC. For the WW* channel the simulation of Ref. |l^ can be 
repeated using the cross section with the fourth generation included. For the rf and ZZ 
channels new simulations in a wide Higgs boson mass interval are necessary. 

The large effect in the gluon fusion discussed can be imitated by other mechanisms, for 
example, in two Higgs doublet models or in A4SSA4 at some values of the mixing angles f3 
and a. In this respect, additional measurements of the Higgs boson production in 77 or ej 
collisions can help to solve our problem. Indeed, adding to the SA4 the fourth generation 
changes strongly the relevant cross sections (see e.g. Fig. ^). Fortunately, the parameters 
which imitate the effect of the fourth generation in the gluon fusion do not give such an 
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imitation for the photon fusion {H'-fj vertex) or the process 67 eH [jT5[. Therefore, 
the study of these processes at photon colhders is very useful to obtain an unambiguous 
conclusion about the existence of next heavy generations. 
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TABLES 



Mh, GeV 


r^, MeV 


MeV 


p4 /p3 
^ qgl qq 


100 


0.093 


0.812 


8.77 


120 


0.155 


1.34 


8.67 


140 


0.241 


2.06 


8.54 


160 


0.357 


3.00 


8.40 


180 


0.507 


4.18 


8.24 


200 


0.702 


5.65 


8.05 


250 


1.46 


10.9 


7.47 


300 


2.87 


19.2 


6.69 


350 


6.28 


33.9 


5.44 


TABLE! Two- 


gluon width for three and four ; 


generations as function of Mh- 


Mh, GeV 






{TlBr^^^)/{Tl^Brl,) 


100 


0.158 




1.12 


120 


0.193 




1.31 


140 


C\ OCA 

0.250 




1.78 


160 


0.403 




3.32 


180 


0.485 




4.00 


200 


0.539 




4.34 


250 


0.710 




5.30 


300 


0.960 




6.50 


350 


1.36 




7.40 



TABLE II. Ratio of the two-photon widths F^^/L^^ and of the cross section gg ^ H ^ 
for four and three generations as function of Mh- 
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Tevatron 


Mh, GeV 


3 generations 




4 generations 


background 


S4/VB 


100 


30 




219 


7400 


2.75 


110 


24 




166 


1740 


4.4 


120 


18 




121 


790 


4.7 


130 


13 




86.5 


456 


4.4 


140 


7 




52 


296 


3.3 




LHC 


Mh, GeV 


3 generations 4 g 


;enerations background 






100 


1170 


8270 


35200 


6.2 44 


5.8 


110 


990 


6330 


8180 


10.5 75 


8.8 


120 


810 


5550 


3640 


13.3 92 


10 


130 


523 


4210 


2085 


11.3 91 


10.2 


140 


402 


2840 


1350 


10.7 78 


8.8 


150 


208 


1610 


946 


6.7 51 


5.9 


160 


31.5 


257 


690 


1.0 8.6 


1.1 



TABLE III. Cross sections of the processes pp ^ . . . + gg ^ H ^ tt for three and four 
generations and for the background pp ^ qq + . . . ^ Z{'y) ^ rf at the Tevatron and the LHC (in 
fb). The S4{fi)/-\/B for the H —>■ decay channel is shown additionally. 
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FIGURES 




FIG. 1. Dominant diagram for Higgs boson production in hadron collisions. 





1 ' ; 

Tevatron 


: / fVbkgd 




: \ 

\ 

- / 


WW ^ 










- /^^'^ 

1 , 1 


f // 



100 150 200 



Mjj, GeV 

FIG. 2. The cross sections of the Higgs boson production for different decay channels calculated 
for the Tevatron. The lower curves correspond to three generations, the upper ones — to four 
generations. The rf background is shown by the dash-dotted line. 
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FIG. 3. Same as Fig. || for the LHC. 
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FIG. 4. The cross section of the Higgs boson production at a photon colhder (averaged over 
the photon spectrum) for different discovery modes: three generations (full line), four generations 
(dotted line), an extra charged vector boson (dashed line). 
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